BMJ Health & Care Informatics

ObjectivesTo determine whether single-layer evaluation characterises clinical risk in the final note of a multilingual ambient AI scribe, and where serious errors arise. DesignTwo-arm evaluation on one common clinical-risk scale, using a frozen, reference-aligned synthetic corpus. SettingScripted consultations spanning 77 languages and five complexity levels, from simple general practice to expert multidisciplinary-team handover. Main outcome measuresPer-session incidence of at least one serious (HIGH or CRITICAL) note-layer error in intrinsic (reference script to note, n=385) and end-to-end (automatic speech recognition (ASR) transcript to note, n=2,302) generation; three LLM raters classified discrepancies using a four-mode taxonomy. ResultsIntrinsic note generation had a 3.4% serious-error rate, with no detectable gradient across complexity (1% to 6%; Cochran-Armitage p=0.42) or language resource (high 2%, medium 5%, low 4%). End-to-end serious errors rose steeply with complexity (L1 1% to L4/L5 24%) and language scarcity (high 7%, medium 10%, low 15%; OR 1.54 per tier, 95% CI 1.15-2.07; p=0.004, GEE clustered on language). Of serious in-note errors, 87% were ASR-derived and 7% note-originated; amplification exceeded correction threefold (fate entropy 1.19 bits). Repeated generation showed moderate reproducibility (Fleiss kappa 0.42); word error rate explained 24% of cross-language variance versus [~]1% for transcription risk density. ConclusionsLow serious-error rates in individual layers did not preclude higher end-to-end note risk. Evaluation should therefore include the clinician-facing note, because component-level metrics cannot fully capture risk created or transformed at the transcription-to-generation interface. What is already known on this topicO_LIAmbient AI scribes are being deployed globally, and safety evaluation has characterised the transcription layer in isolation, most commonly with word error rate C_LIO_LIFrequency-based transcription metrics such as word error rate do not directly encode clinical consequence, and component-level performance may not represent the risk of the final generated note C_LIO_LILarge language models produce clinically relevant output stochastically, so identical prompts can yield materially different content C_LI What this study addsO_LINote generation from a perfect transcript had a low observed serious-error rate and no detectable complexity or language-resource gradient, whereas the end-to-end system showed steep gradients on both axes. C_LIO_LISerious note-layer errors were predominantly ASR-derived, but note generation transformed upstream errors variably, amplifying serious errors about three times more often than it corrected them C_LIO_LITranscription-layer metrics had limited predictive value for note-layer risk after accounting for complexity and language-level dependence; word error rate explained approximately one-quarter of cross-language variance C_LI How this study might affect research, practice or policyO_LIEvaluation and procurement of ambient scribes should include end-to-end assessment of the clinician-facing note, with results stratified by consultation complexity and language where relevant C_LIO_LIA three-surface root-cause alignment using a four-mode root-cause taxonomy (note-originated, propagated-ASR, amplified-ASR and silent-correction) can attribute documentation errors to their point of origin or transformation C_LIO_LILanguage-related performance should remain a surveillance target because disparities can emerge after components are composed, even when corresponding gradients are not detectable in an individual layer C_LI

Background: Consumer use of AI chatbots for health advice is rising, yet triage safety relative to established services remains unclear. Australia's Healthdirect, a government-backed symptom checker with 2.4 million uses in FY2024-25, remains unevaluated against frontier large language models (LLMs), and whether premium subscriptions improve triage safety remains unexplored. This study compared the triage accuracy and safety of Healthdirect against six LLM configurations across ChatGPT, Claude, and Gemini, assessed whether paid subscriptions improve triage safety, and characterised each system's error patterns. Methods: Forty-five clinical vignettes from the Semigran et al. benchmark spanning emergency, non-emergent, and self-care categories (15 each) were evaluated across seven systems. Healthdirect was tested following a seven-rule interaction protocol. LLMs were evaluated using first-person patient-language prompts under free-tier and paid-tier conditions. Outcomes were triage accuracy, emergency sensitivity, under-triage, and critical misses, analysed using Cochran's Q, Bonferroni-corrected McNemar tests, Cohen's kappa, and Wilson intervals. Findings: Triage accuracy differed significantly (Cochran's Q = 36.79, p < 0.001). Healthdirect achieved 48.9% accuracy (95% CI 35.0% to 63.0%; kappa = 0.233) versus 73.3% to 86.7% for LLMs (kappa = 0.600 to 0.800). Healthdirect operated under conservative interactive defaults while LLMs received complete information in a single prompt, which may have disadvantaged Healthdirect. Emergency sensitivity was 46.7% versus 80.0% to 86.7% for LLMs. Healthdirect produced two critical misses; no LLM produced any across 270 evaluations (95% CI 0% to 1.4%). When LLMs undertriaged, they recommended GP care rather than self-care. No tier differences were significant (all p > 0.05), and most systems over-triaged self-care cases. Interpretation: Frontier LLMs demonstrated higher triage accuracy and safer error profiles than Healthdirect. All LLMs avoided critical misses; Healthdirect did not. Premium subscriptions did not significantly improve triage safety. These findings support clinical governance decisions about whether LLMs warrant formal evaluation alongside government-backed symptom checkers.

Background: As required by the 21st Century Cures Act, all test results, including final radiology reports, are released to patients as soon as the results are finalized. However, radiology reports are primarily intended for communication between medical professionals and can often be difficult for patients to interpret, particularly in cancer care settings. We developed a tool that leverages a large language model (LLM; ChatGPT-5.2, OpenAI, San Francisco, CA) to draft a patient-friendly plain language version of imaging results (Patient-Friendly Scan Result Generation [PFSG] tool) that oncologists can incorporate into their communication with patients via patient portals. This article describes our mixed-methods study protocol. Objective: Our primary aims are to: 1) assess whether patients who receive patient-friendly versions of scan results report better understanding of their results than patients who do not; 2) assess whether introduction of the PFSG tool affects provider workload; and 3) explain how and under what circumstances the PFSG tool enhances communication and workflows, and affects patient and provider outcomes. Methods: We will use an embedded mixed-methods, prospective cohort design within a quality improvement (QI) initiative. The PFSG tool will be implemented using a modified three-wave staircase trial design, with providers in randomized cohorts of oncology disease groups progressively gaining access to the tool over the course of the study. Approximately 60 providers from 11 oncology disease focus groups will be randomized. Longitudinal provider outcomes will be measured through surveys at one month pre-implementation, one month post-implementation, and three months post-implementation. Patient outcomes will be measured cross-sectionally through surveys at one month pre-implementation (n=350) and at post-implementation result point-of-contact (n=350). Additionally, we will conduct interviews with patients and oncologists to inform the iterative design of the tool and its implementation. Results: This study was funded in September 2025 and data collection began in February 2026. As of September 2026, the PFSG tool has been rolled out to all oncology providers, with Cohort 3 post-implementation data collection beginning in August 2026. A total of 738 patient baseline surveys have been collected and patient post-implementation data collection is in progress. Conclusions: This is the protocol for a study of the implementation of a patient-friendly results tool in oncology provider workflows to facilitate improved communication of radiology results to oncology patients. The results of this study will lend evidence to the utility and acceptability of the study tool, and tools like it, to close an important oncology care gap. If the tool is shown to improve key oncology patient and provider outcomes, it may lead to development of a wider release for more general patient populations.

Background. Dedicated allied health professions (AHP) informatics posts remain rare and largely unevaluated: one NHS trust in ten reports a formal AHP informatics lead. Objectives. To evaluate stakeholder-perceived value and experience of a dedicated AHP Information Officer (AHPIO) role supporting an organisation-wide electronic health record (EHR), and to derive a working framework of the role's practice. Methods. Cross-sectional online survey within a registered service evaluation at a specialist cancer centre in England. A purposive census of 40 stakeholders (clinical and operational AHP staff, digital champions and EHR programme colleagues) rated six agreement items with four free-text questions. Analysis led with distributions, medians and Wilson intervals; inference was exploratory; free-text coding is fully audit-trailed. Results. Twenty-seven stakeholders responded (68%). Five items sat at a ceiling (medians 4 to 5; agreement 95% to 100%); all 25 answering the overall item agreed the role adds value, 80% strongly. Data and reporting access was the outlier (median 3.0; 52% neutral or below; agreement 48%, 95% CI 28% to 68%), ranking below every other item (Friedman chi-squared 36.3, df 5, 20 complete cases; Holm-adjusted post-hoc comparisons, rank-biserial -0.73 to -1.00). Nine themes located value in brokerage and routing (18/26), concrete outcome accounts (16/26) and a clinician who understands AHP work (12/26). Conclusions. Stakeholders located the role's value in brokerage, clinical-digital translation, representation and outcome response, resting on a named trusted contact; self-service data is the part-built pillar and development priority. The framework and method offer a replicable template for evaluating AHP informatics roles.

Importance. Cancer is the second leading cause of death among patients in the Caribbean, where outcomes are associated with delayed clinical navigation to screening, diagnosis, and treatment. Artificial intelligence is increasingly used to guide patients with cancer to care, but whether these systems provide clinically actionable, facility-verified guidance for individuals in this population, and whether governance of the system is associated with the quality of that guidance, has not been evaluated. Objective. We tested whether a governed community learning platform navigates Caribbean cancer patients better than four ungoverned AI systems, and we tracked how community intelligence accumulates over time. Design, Setting, and Participants. We deployed a community learning ledger (CaribChat.ai) across ten Caribbean jurisdictions beginning March 2, 2026, and report all sessions through June 1, 2026 (N=207). An initial actively-promoted accrual period (March 2 - April 6, 2026; 168 sessions) was followed by continued organic use after active clinical promotion ceased. We then submitted the same 28 patient screening queries to ChatGPT (GPT-4o), Claude Haiku 4.5, DeepSeek-Chat, and OpenEvidence on April 5-6, 2026. Claude Haiku 4.5 powers CaribChat; testing it without governance isolates the governance effect. The platform requires no registration. Exempt under 45 CFR 46.104(d)(4)(ii). Main Outcomes and Measures. We classified 207 community sessions by thematic domain and temporal phase. We scored each of five systems on Caribbean facility citation, actionable navigation, and US-resource leakage across 28 screening queries. Results. The ledger accumulated 207 sessions - 168 during an actively-promoted accrual period (March 2 - April 6) and 39 after active clinical promotion ceased. Community engagement evolved from screening questions to active treatment navigation and diaspora engagement. CaribChat cited verified Caribbean facilities in 28/28 (100%) responses versus 10/28 (35.7%) for ChatGPT and 9/28 (32.1%) for OpenEvidence. CaribChat provided actionable navigation in 28/28 (100%) versus 2/28 (7.1%) for OpenEvidence (P<=.001). The same model scored 100% with governance and 54% without (P<=.001). DeepSeek cited US resources in 57.1% of Caribbean responses. After active clinical promotion ceased, off-codebook queries rose from 2.4% to 26.7% across phases while the governance contract continued to reject every adversarial probe - the community persisted but drifted from the cancer codebook absent clinician curation. The deployment operated within the OECS Health Strategy 2030 and CARICOM regional health frameworks, with queries originating across Caribbean jurisdictions led by Trinidad and Tobago. Conclusions and Relevance. Every ungoverned AI system we tested failed Caribbean cancer navigation. The best scored 68%. The most widely adopted physician platform scored 7%. The same foundation model scored 100% with governance and 54% without. Community intelligence accumulated from the population it serves, not published literature, is what makes health AI work in SIDS. The post-promotion decay shows the requirement is bidirectional: sustained, on-codebook engagement depends on patients and clinicians working together - community participation and active clinical curation are jointly necessary for maximum AI leverage.

Background Digital legacy (the digital information available about someone following their death) has increasing societal importance as personal assets and interactions become increasingly digitized. Healthcare professionals often have a limited understanding of how to address digital legacy in practice, and there is a lack of interdisciplinary networks to improve education, research, and professional development in digital legacy. Objective This paper describes the development of an interdisciplinary initiative designed to build research capacity and develop consensus-based recommendations for integrating digital legacy into palliative care. Method Over 12-months, we conducted interdisciplinary engagement activities with diverse stakeholders, including clinicians, designers, and sociologists. We used a modified World Cafe method to facilitate dialogue and capture feedback on how memories are digitally curated, the management of digital estates, and intergenerational perspectives on digital legacy. Results We identified eight core recommendations for research and policy, including promoting digital legacy education, supporting policy development, and broadening the scope of interdisciplinary research. Our discussions highlighted the complexity of modern digital estates and the need for legal and ethical frameworks to protect individual rights. Conclusions The Network demonstrates that interdisciplinary collaboratives can address important issues relating to digital legacy, which provides a foundation to conduct collaborative research that improves the management of digital legacies in society.

Background: Breaking bad news (BBN) is an essential part of healthcare communication especially in the fields of oncology. palliative care and life threatening illness. Objective: This study aims to analyze the global research trends, publication patterns and the intellectual structure of BBN literature from 2000 to 2025. Methods: Bibliometric study was performed on Scopus database. A total of 843 publications were retrieved and analyzed by bibliometric techniques. Visualization tools such as VOSviewer and R Studio were employed, along with Lotkas and Bradford laws. Results: The findings indicate that there is a steady growth of publications. The United States emerged as the leading contributor. Journals including Patient Education and Counseling were dominant in the field. Key areas of focus identified through thematic analysis included oncology, palliative care and communication, while emerging themes included empathy and telemedicine. Conclusion: BBN research has evolved into a mature, interdisciplinary field with increasing emphasis on patient centered communication.

Objectives: To develop a multidimensional framework for assessing organisational and system readiness for federated health-data services, and to report its formative application across three heterogeneous UK ecosystems. Methods: The Health Data Readiness Level (HDRL) framework was developed from a structured landscape review of 56 maturity and readiness frameworks, first-principles requirements analysis, artificial-intelligence-assisted synthesis with human source verification, and stakeholder refinement. It comprises 64 indicators in eight domains and five ordered levels. Formative application examined three heterogeneous UK health-data research ecosystems using documentary evidence, professional stakeholder input, workshops, a structured right-of-reply process, cross-case calibration, and two illustrative research use cases. Analysis was descriptive; the application was not designed as psychometric validation or a league table. Results: All 64 indicators were scoreable in each case. Publicly reported profiles ranged from Developing (Level 2 -3) to Managed (Level 3 -4); all three cases met the proposed minimum for five foundational indicators. Recurring constraints concerned evidence of measured service performance, national-scale primary-care data access, cross-jurisdiction governance reciprocity, workforce capacity, and sustainable funding. Pandemic-era four-nation research was delivered through coordinated local analyses and meta-analysis, rather than routine automated federation. Discussion: HDRL operationalises a broad service- and system-readiness view that complements technical and governance specifications. Content validity, inter-rater reliability, aggregation choices, responsiveness and predictive validity remain to be established. Conclusion: HDRL is an evidence-informed candidate improvement and planning instrument for federated health-data services. It should not yet be used as an accreditation standard or as an official participation threshold for the UK's Health Data Research Service.

BackgroundEvaluating fairness in artificial intelligence (AI) is essential before clinical implementation to prevent unintended harm across diverse patient populations. This study assesses performance differences of an image-based surgical site infection (SSI) detection model across age, gender, geography, race, and skin color. MethodsModel development/validation study using retrospective, multicenter data across surgical subspecialties from adults who submitted an image of their incision via the electronic health record within 30 days of their procedure between 2019-2024. Area under receiver operating curve (AUROC) was compared by age (<55 vs. [&ge;]55 years old), gender (male vs. female), geography (Rochester, Minnesota vs. Jacksonville, Florida vs. Scottsdale, Arizona vs. regional Minnesota/Wisconsin areas), race (White/Caucasian vs. non-White/Caucasian), and skin color (Fitpatrick I-III vs. IV-VI and Monk A-E vs. F-J). ResultsTraining data contained 13,702 images from 4,274 patients between 2019-2022 (50.1% [&ge;]55 years old; 63.2% female; 91.6% White/Caucasian; 58.5% Rochester, 17.3% Florida, 10.5% Arizona; 96.3% light skin tone), of which 1,348 images (9.8%) had an SSI. No differences in SSI detection existed for age, gender, race, or skin color (AUROC 0.83-0.87, p>0.09), however geographical differences were significant: lowest AUROC 0.83 (95% CI 0.77-0.88) vs. highest AUROC 0.87 (95% CI 0.84-0.89, p=0.01). Temporal validation using 4,242 images from 1,353 patients between 2022-2023 showed intact fairness across subgroups ({Delta}AUROC [&le;]0.06) except geography ({Delta}AUROC [&le;]0.16). ConclusionsThe model behaved fairly across age, gender, race, and skin color, however geographical differences persisted. These findings have practical implications underscoring the importance of evaluating bias in clinical AI before deployment.

As health AI tools enter clinical spaces, how they are perceived by providers in safety-net environments shapes whether their benefits reach underserved populations. We examined the degree and drivers of healthcare providers' trust in health AI, whether that trust moderates perceived benefits for safety-net populations, and how providers balance benefit against concern across four specific hypothetical AI tools. We used a two-study mixed-methods design: 18 semi-structured interviews with healthcare practitioners and research leaders, followed by a survey of 229 Texas healthcare providers incorporating four vignettes describing hypothetical AI tools, each rated for appeal and concern and followed by open-ended explanation. Responses were thematically coded and stratified by self-reported trust in health AI (47.6% high trust, 52.4% low trust). Low-trust respondents most often cited building trust (50.5%) and data bias and accuracy (33.0%) as drivers of their current trust level, while high-trust respondents more often cited efficient and effective care (23.5% vs. 3.7% low-trust, p<0.001) and accuracy of data (11.8% vs. 1.8% low-trust, p=0.004); concerns about data security, governance, and the need for human oversight were raised at similar rates by both groups. When asked how AI could specifically benefit safety-net populations, both groups converged on expanding access to care (45.9% each), and none of 25 benefit-related codes differed significantly by trust group. Across all four vignettes, mean appeal ratings exceeded mean concern ratings, and high-trust providers rated every tool as more appealing and, for three of four tools, less concerning, without changing which tool ranked where relative to others. Providers exhibit conditional optimism: support is contingent on governance, human oversight, and context-sensitive implementation rather than on the technology itself. Realizing AI's promise for safety-net populations will require human-in-the-loop models, localized implementation strategies, community-informed design, and engaging providers directly in policy design and tool development.

Background. Clinical quality measurement often relies on manual abstraction of medical records, an approach that is costly, burdensome, and often infeasible for measures requiring interpretation of narrative text; these constraints have shaped measure development itself, filtering out clinically important measures that are too difficult to operationalize. We evaluated whether neuro-symbolic artificial intelligence (NSAI), which combines large language model extraction with symbolic reasoning, could reliably abstract complex quality measures from narrative pathology reports. Methods. The NSAI system decomposes each measure into atomic questions and is aligned to real-world reports through case-based refinement, an iterative human-in-the-loop process. Using 2,000 independently double-abstracted reports, we compared NSAI-based abstraction against trained human abstractors across four pathology quality measures established by the College of American Pathologists. Results. The NSAI system's agreement with the adjudicated gold standard (Cohen's kappa = 0.95) matched or modestly exceeded that of the trained human abstractors measured against the same standard (kappa = 0.92), with particularly strong performance on Gastrointestinal Metaplasia (CAP 43). In component analyses, case-based refinement drove the largest accuracy gains (up to 0.25 in kappa), whereas architectural decomposition primarily reduced performance variance across language-model backends more than tenfold, a property essential for clinical deployment. Conclusions. These findings suggest that automated abstraction could enable census-level quality measurement, reduce reporting burden, and expand the range of clinically meaningful measures that can be operationalized from narrative clinical documentation.

The explosive growth of ambient AI scribes in healthcare has focused almost exclusively on relieving clinician documentation burdens and enhancing clinician efficiency and reducing burnout. While most research has focused on the clinician experience, these same types of tools, such as a patient held AI Personal Health Assistant, can be put in the hands of patients to facilitate communication with healthcare providers, to provide greater autonomy and activation for patients in managing their own healthcare, and to facilitate the task of receiving and retaining complex new information. However, to ensure these tools serve as more than just administrative aids, we partnered with a large technology company to conceptually codesign how such a tool might be studied in real world settings in a way that reflected outcomes of greatest importance to patients and families. We used the Nominal Group Technique and engaged thirty three patients and caregivers and ten multidisciplinary clinicians in structured virtual sessions to identify the outcomes on which subsequent studies should hinge. Our findings reveal a significant misalignment between current industry trends and patient priorities. While the market emphasizes the scribe function for billing, stakeholders overwhelmingly prioritized the AIPHAs role in longitudinal care management and post visit coordination. The most critical outcome identified was the impact on communication and respect, suggesting that patients view AI as a mechanism to improve the human to human dynamic and ensure a shared understanding of the care plan. Our findings further highlighted that the primary hurdle for the use of patient held AI is not technical utility but clinician trust and the need for formal consent processes. Importantly, as this technology is new and integration into the clinical experience is still being explored, patients cannot assume they will have access to the transcript of their own encounter when that record is created and held by the health system. This gap highlights the potential value of a patient held tool, preserving discussions from their visit and follow up steps in clearer detail than they may receive in their clinical note or traditional visit summary. From a system perspective, this transition requires a strategic shift from measuring workflow focused efficiency to measuring patient side advocacy and engagement. By centering the evaluation of AI on these patient defined metrics and fundamentally designing subsequent AI investigations using a partnership model, health systems can move beyond simple automation toward a model of care delivery that empowers patients to navigate complex clinical journeys with greater autonomy and clarity.

Objectives: To assess the feasibility and early performance of an AI-assisted screening workflow that reviews inpatient discharge summaries to identify potential treatment injuries for clinician review and, where appropriate, Accident Compensation Corporation (ACC) treatment injury claim consideration in New Zealand. Design: Single-site, six-month pilot reported as a quality-improvement/service-evaluation activity. A large language model (LLM)-enabled system acted as an index screening test over discharge summaries to generate candidate treatment injury signals for structured clinician adjudication and escalation. Setting: A New Zealand district hospital (Wairarapa). Screening operated on routinely produced inpatient discharge summaries. Participants: All inpatient discharge summaries produced during the pilot period were screened (n = 2,642). The unit of analysis was one discharge summary per inpatient episode. Intervention: An LLM-based screening workflow that outputs structured candidate harm signals with supporting excerpts and a certainty label. Outputs were reviewed by clinicians using a structured adjudication template. Suitable cases could be escalated to senior clinicians and followed through the local ACC treatment injury process. Main outcome measures: Primary outcomes were yield, positive predictive value (PPV) of AI- flagged signals for clinician-adjudicated iatrogenic harm, and the rate of unsupported/hallucinated outputs. Secondary outcomes included overlap with incident reporting and administrative coding, downstream ACC claim outcomes, and clinician review workload. Results: Across 2,642 inpatient discharge summaries screened, the AI-assisted workflow generated 237 candidate signals (yield 9.0%; 95% CI 7.9% to 10.1%). Clinician adjudication classified 102 as confirmed iatrogenic harm, 43 as possible harm requiring additional context, 81 as no harm, and 11 as unsupported/hallucinated outputs. PPV for confirmed harm was 43.0% (95% CI 36.8% to 49.4%). PPV for confirmed-or-possible harm was 61.2% (95% CI 54.9% to 67.1%). Seven ACC treatment injury claims were lodged as a direct result of AI-assisted surfacing; three were accepted, two declined, and two were held/pending at analysis. Organisational ACC data identified 68 treatment injury claims lodged during the same period, of which 17 overlapped with AI-identified cases. Conclusions: AI-assisted screening of discharge summaries can feasibly increase detection of potential treatment injuries for structured clinician review and may support more systematic ACC consideration and safety learning. Safe scaling requires transparent reporting of model and prompt details, robust human adjudication and escalation pathways, and governance controls to mitigate false positives and unsupported outputs. A next-stage evaluation should estimate diagnostic accuracy against a chart-review reference standard and examine implementation outcomes including workload, acceptability, fidelity and sustainability.

Objective: To develop and evaluate a framework for characterizing clinician editing of Artificial Intelligence (AI)-generated[KS1.1] documentation and assess its feasibility for health system-level monitoring of AI scribes. Materials and Methods: We analyzed outpatient encounters in which an AI scribe was used at a single academic health system, examining the History of Present Illness (HPI) and Assessment and Plan (A&P) sections of notes. We characterized edits on three dimensions[KS2.1][AG2.2]: lexical edit intensity based on Levenshtein distance, embedding edit intensity BERT[KS3.1]Score, and clinical edit intensity based on removed and added UMLS[KS4.1][AG4.2] concepts.[KS5.1][AG5.2] We analyzed the Positive Predictive Value (PPV) of clinical edit intensity as a measure of clinically meaningful editing using clinicians as the gold standard, examined correlations among dimensions, and designed exponentially weighted moving-average control charts to monitor longitudinal changes in clinician editing behavior.[KS6.1][AG6.2] Results: 268,379 encounters were included (267,654 HPI, 267,594 A&P). Clinical edit intensity [&ge;]1 had an 88.9% PPV for clinically meaningful editing. Lexical and embedding edit intensity were highly correlated (Spearman {rho} 0.95), while clinical edit intensity was less strongly correlated with both ({rho} 0.77-0.81). Longitudinal monitoring detected changes coinciding with system-wide rollout.[KS7.1][AG7.2] Discussion:[KS8.1][AG8.2] Clinicians edited A&Ps more heavily than HPIs, potentially reflecting greater attention to content involving clinical decision-making. Over one-third of sections involved clinical concept changes, and clinical edit intensity identified clinically meaningful edits while providing information complementary to lexical editing measures. Conclusion: Clinician editing can be characterized at scale using complementary editing dimensions, providing a scalable signal for post-deployment surveillance of the human-AI documentation process.

Conversational systems based on large language models may support point-of-care evidence retrieval, but most evaluations use static benchmarks and few involve practicing physicians in low- and middle-income settings. We assessed the validity of answers produced by physicians using an evidence-traceable conversational system versus usual information-search practice, alongside response efficiency and acceptability. Physicians in Latin America were randomly assigned to answer four simulated clinical cases, four open-ended questions each, with system support or usual search practice. Two blinded specialists per area scored each response with a six-dimension rubric. The main outcome was the composite validity score aggregated per physician, analyzed with logistic regression adjusted for academic degree. Of 202 physicians who began, 71 completed and were analyzed, 26 supported and 45 using usual practice, yielding 1,136 responses. Physicians using the system scored higher, with medians of 2.83 versus 2.46 (difference 0.38; 95% CI 0.17 to 0.54; P < .001). In the adjusted model they were more likely to meet the validity threshold (odds ratio 3.61; 95% CI 1.16 to 11.17; P = .026). In the adjusted models the association held for the four criteria with the higher interrater agreement, and did not reach significance for accuracy alone. Response time and self-reported searches did not differ, though uneven missingness makes that comparison uninformative. Acceptability among supported physicians was high (mean total score 2.92 of 3). An evaluator outside the organization repeated the accuracy-only analysis, which showed no significant difference; the composite was designated the main outcome after that replication, so the three specifications are reported side by side. The analyzed sample comprises volunteers who completed the exercise, so the findings are exploratory.

Background: Public facing large language models (LLMs) are increasingly used for health guidance, including triage recommendations. We evaluated whether augmenting LLM prompts with standardized clinical concepts from the Unified Medical Language System (UMLS) could improve the safety and robustness of clinical triage recommendations. Methods: We used a publicly available dataset comprising 60 clinician-authored clinical vignettes, each represented in 16 demographic and narrative variations, yielding 960 vignette-factor combinations. Clinical entities were extracted using a two-stage pipeline combining ClinicalBERT-based named entity recognition with rule-based identification of laboratory abnormalities. Extracted entities were mapped to UMLS Concept Unique Identifiers (CUIs). Negated concepts were excluded. A confidence-weighted CUI voting classifier was trained using empirical associations between CUIs and clinician-assigned triage categories. We compared five approaches: CUI-only classification, MedGemma 27B, MedGemma 27B augmented with CUIs, GPT-4o-mini, and GPT-4o-mini augmented with CUIs. Outcomes included overall accuracy, under-triage, over-triage, emergency-case accuracy, and sensitivity to anchoring statements. Results: CUI augmentation decreased under-triage but increased over-triage in both models tested (GPT-4o-mini and MedGemma 27B). It improved high-acuity recognition while reducing recognition of low-acuity cases. CUI augmentation had mixed effects on overall triage accuracy; accuracy increased for MedGemma 27B but decreased for GPT-4o-mini. Emergency-case accuracy improved from 73.0% to 80.7% for GPT-4o-mini and from 60.5% to 68.5% for MedGemma 27B. CUI augmentation also reduced susceptibility to anchoring statements. These findings suggest that the principal value of CUI augmentation may be shifting model behavior toward safety-oriented behavior rather than uniformly improving overall accuracy. Conclusion: Ontology-grounded prompt augmentation shifted LLM triage recommendations toward greater sensitivity to high-acuity presentations and reduced overall under-triage. These safety gains were accompanied by increased over-triage and mixed effects on overall accuracy. A hybrid architecture combining LLM-based language understanding with interpretable UMLS-derived clinical concepts may improve the safety and robustness of AI-assisted triage. Further evaluation using real-world patient communications and clinical outcomes is warranted.

Background: Electronic health records (EHRs) are central to NHS clinical practice, and the value of the data they hold depends on clinicians entering structured data. Structured data entry (SDE) is nonetheless used inconsistently, and most evidence comes from US settings documenting barriers rather than the mechanisms interventions must target. This is, to our knowledge, the first study to apply the Theory of Planned Behaviour (TPB) to SDE, to examine SDE behaviour among UK NHS doctors, and to apply corpus linguistics to clinicians' free-text accounts of EHR use. Methods: A two-part mixed-methods study was conducted at a large NHS teaching hospital in central London using the Epic EHR. Study 1 was a cross-sectional survey of non-consultant grade doctors comprising the System Usability Scale (SUS), a purpose-designed TPB questionnaire, and two open-text questions analysed using corpus linguistics. Study 2 was an Interpretative Phenomenological Analysis (IPA) of semi-structured interviews with six resident doctors. Results: Eighty-three doctors responded; 65 completed the SUS and 49 the TPB questionnaire. Mean SUS score was 62.1 (95% CI 58.3-65.9), below the benchmark of 68 and equivalent to a grade of "D". Structured features were used markedly less often than narrative notes. TPB constructs explained 56% of the variance in intention (F(3,45) = 18.95, p < .001); attitudes (p = .003) and subjective norms (p = .009) predicted intention independently, whereas perceived behavioural control did not (p = .14). Intention predicted self-reported use of problem lists, event-specific templates and standardised documents, but not narrative notes. Corpus analysis identified time pressure and interface friction as dominant barriers. IPA generated four themes: structured documentation as cognitive relief; as fragile collective infrastructure; as a practice clinicians are expected to perform but not enabled to learn; and as a format that can flatten clinical reasoning. Conclusions: Engagement with SDE is driven principally by attitudes and social norms rather than by perceived control, and the barriers clinicians describe arise from the intersection of interface design, training, team norms and institutional expectation. Interventions addressing usability alone are unlikely to succeed: sustained change requires team-level normative reinforcement, specialty-relevant training that explains both how and why, and systems that preserve space for clinical narrative. Keywords: Electronic health records; structured data entry; Theory of Planned Behaviour; System Usability Scale; Interpretative Phenomenological Analysis; corpus linguistics; clinical documentation; data quality; NHS

Background: Large language models (LLMs) that use retrieval-augmented generation (RAG) are increasingly used to answer clinical questions, although the evaluation of these systems remains limited. Building on previous studies conducted by our team, this case report aimed to improve upon this knowledge gap by applying a reusable methodology to compare the performance of eight LLMs that utilize RAG techniques for evidence synthesis. Case Presentation: Eight commercially available RAG LLM tools (OpenEvidence, Undermind, Consensus, SciSpace, Elicit, MediSearch, EvidenceHunt, and Scite) were evaluated using twelve ChatGPT-generated clinical questions on the topics of treatment, etiology, and prognosis. To enable comparison, we prompted ChatGPT to identify all key unique medical concepts from the full set of LLM responses to each question. Concepts were categorized as critical ("must-have") or non-critical ("nice-to-have") for answering the clinical question. Experienced information scientists were consulted at each step for their expertise. Descriptive statistics and Kruskal-Wallis tests were used to compare performance across tools and question categories. No significant differences were found among the eight RAG LLMs in their coverage of "must-have" (p=0.95) or "nice-to-have" (p=0.16) key unique medical concepts, and no single tool consistently captured all identified concepts. Conclusions: These findings suggest that RAG LLMs may be supplementary tools for evidence retrieval and synthesis but cannot, at this time, fully replace comprehensive expert review of the medical literature. The evaluation framework presented here may be a useful model for future comparative assessments of rapidly evolving AI evidence synthesis tools.

Background: The uptake of new digital and AI based technologies in the health care setting, including digital diagnostics, has been comparatively slower than in other sectors. This is a consequence of the complex legislature, lack of routine procedures for the secondary use of sensitive medical data. Inadequate level of education and incentives for different stakeholders prevents overcoming strong barriers, such as clinical workflow integration, and need for comprehensive eval uation of utility of interventions which requires multi-disciplinary collaboration effort. The paper presents a Re-hospitalization Challenge, organised by the European Digital Innovation Hub (EDIH)- AI4Health.Cro, a co-creation activity and an effective tool for overcoming the mentioned barriers for implementation of new AI technologies and overall innovation ecosystem build-up. Methods: Crowdsourcing challenges or hackathons are R&D strategies in which ad-hoc assembled competing teams collaborate on early-stage technology devel opment or research problem solving. Re-hospitalisation challenge, in our case, has had multiple facets: as a co-creation activity tasked with construction of an AI predictive model with the explanatory support and plausible business model for its deployment; it involved also extensive organisers effort: data anonymisation, provision of secure processing environment, and comprehensive assessment of solutions. The paper details the challenge structure and processes, from problem inception to execution and comprehensive evaluation of developed solutions. Results: As a co-creation effort in digital healthcare, the challenge provided results on multiple levels: (i) through tailored solutions based on real evi dence data, serving as benchmarks or prototypes for later deployment in practice; (ii) through sandboxing EHDS principles to enable secondary data use; (iii) as an ecosystem building event yielding ad-hoc collaborative teams and cross-disciplinary participative learning experience. Conclusions: Crowdsourcing innovation challenges are valuable digital health care ecosystem building tool that provides a number of direct and indirect 2 impacts, from providing valuable diagnostic prototypes for innovation elicitation, creation and transformation of startups, to streamlining secondary use of clinical data for research and innovation. Overall impact is also enhanced through raising the awareness and education of all stakeholders relevant for the translation of AI technologies and into routine clinical setting.

Background Virtual hospital (VH) pathways support early discharge through remote monitoring, but limited evidence has hindered implementation in colorectal surgery. This study aimed to define patient- and carer-relevant outcomes and experiences of VH following colorectal surgery. Methodology A patient and public involvement and engagement (PPIE) consultation was conducted with 8 participants (7 patients, 1 carer; 4 women, 4 men) who had experienced VH following bowel resection at a high-volume robotic unit. Purposive sampling ensured that 50% of participants had experienced readmission. The 90-minute session was delivered via Microsoft Teams. Data were analysed using reflexive thematic analysis. Results Seven themes were identified: readmission, remote monitoring, carer burden, recovery, equity, readiness for discharge, and information delivery. Patients supported early discharge when remote monitoring enabled timely detection of complications and readmission pathways were efficient. Readmission was not perceived as failure but as appropriate escalation. Dissatisfaction with readmission was related to delays in emergency care. Remote monitoring provided psychological safety, with patients feeling held at home. Carers assumed substantial, often unrecognised, quasi-clinical roles. Recovery was defined by return to function rather than length of stay. Equity concerns were evident, with VH favouring those with adequate support at home, digital literacy, and language proficiency. Discharge readiness was both clinical and psychological. Information delivery at discharge was often poorly retained and requires reinforcement preoperatively at every encounter with patients and carers. Conclusions VH pathways are acceptable and valued. Readmission is a marker of system responsiveness rather than failure of early discharge on VH. Psychological preparedness, carer support, and equitable access are critical to successful and scalable implementation of early discharge using a virtual hospital.